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ABSTRACT 

A study sought, to identify the engaging 
characteristics of computer- assisted instruction (CAI) and to 
determine why engagement in CAI programs remains higl?, whether 
engagement can be increased experimentally by changing the level of 
difficulty of the lessons, whether engagement is highest when lessons 
match the student's level of competence, and whether engagement and 
disengagement scores are related. Subjects were 4 0 fourth graders who 
had been working with a drill-and— practice mathematics CAI program 
since first grade. Five lessons of varying difficulty were presented 
to the students. Their behavior while working on the lessons was 
videotaped and classified. The engaging characteristics of CAI that 
were identified were: its curiosity provoking aspects, its immediate 
feedback, its provision of a form of competence testing for the 
student, and its presentation of lessons matched to the student's 
level of competence. Engagement was highest on easy lessons and did 
not begin to drop until the subjects missed more than 20S5 of the 
problems per lesson. The maximum point of engagement could not be 
dtereirined. Measures of engagement and disengagement were found to 
vary somewhat independently. (JY) 
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Introductory Statement 



The Center is concerned with the shortcomings of teaching in 
American schools! the ineffectiveness cf many American teachers in 
promoting achievement of higher cognitive objectives, in engaging 
their students in the tasks of school learning, and, especially, in 
serving the needs of students from low-income areas* Of equal con- 
cern is the inadequacy of American schools as environments fostering 
the teachers’ own motivations, skills, and professionalism. 

Tht*. Center employs the resources of the behavioral sciences — 
theoretical and methodological— in seeking and applying knowledge 
basic to the achievement of its objectives. Analysis of the Center's 
problem area has resulted in three programs: Heuristic Teaching, Teach- 
ing Students from Low-Income Areas, and the Environment for Teaching. 
Drawing primarily upon psychology and sociology, and also upon 
economics, political science, and anthropology, the Center has formula- 
ted integrated programs of research, development, demonstration, and 
dissemination in these three areas. In the Heuristic Teaching 
program, the strategy is to develop a model teacher training system 
integrating components that dependably enhance teaching skill. In the 
program on Environment for Teaching, the strategy is to develop 
patterns of school organization and teacher evaluation that will 
help teachers function more professionally, at higher levels of morale 
and commitment. In the program on Teaching Students from Low-Income 
Areas, the strategy is to develop materials and procedures for engag- 
ing and motivating such students and their teachers. 

This report describes some of the relevance of educational tech- 
nology for instruction in low-income areas. 
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Abstract 



The purpose of this study was to identify the engaging character- 
istics of computer-assisted instruction, and to determine why engage- 
ment in CAI programs remains high, whether engagement can be increased 
experimentally by changing the level of difficulty of the lessons, whether 
engagement is highest when lessons match the student’s level of competence, 
and whether engagement and disengagement scores are related. Subjects 
were 40 fourth graders who had been working with a drill-and-practice 
routine since first grade. Five lessons were presented to each student, 
in a predetermined order, by a teletype similar to the one they had used 
in the elementary school. The students’ behavior, while working on the 
lessons, was classified in five categories and was recorded on videotape 
every five seconds. Scores for engagement and disengagement were recorded 
per minute after each observation and graphed after each lesson. Because 
of faulty videotape equipment, it was only possible to include nine sub- 
jects in the final study; results were therefore presented graphically 
rather than statistically. 

The engaging characteristics of CAI that were identified are (a) its 
curiosity-provoking aspects, (b) its immediate feedback, (c) its pro- 
vision of a form of competence testing for the student, and (d) its pre- 
sentation of lessons matched to the student's level of competence. En- 
gagement was high on easy lessons and did not begin to drop until the 
subjects missed more than 20 percent of the problems per lesson. The 
maximum point of engagement could not be determined. Measures of en- 
gagement and disengagement were found to vary somewhat independently. 

The experimenters still predict that there are levels of both ease and 
difficulty at which engagement should decrease. 



iv 



o 



4 



Contents 



The Math Drill-and -Practice Routine 

The Concept Block Approach 

Distinguishing Characteristics of CAI . 

The Process of Engagement 

Initial Engaging Feacures of CAI. . . . 
Features of CAI that Maintain Engagement 
Method 



Description of Experimental Lessons 11 

Development of the Observation Instrument 17 

Categories of Engagement and Disengagement 18 

Indices and Variables 21 

Results 29 

Findings 31 

Conclusions 36 

References 39 



v 



*. 





- 



THE EFFECT UPON STUDENTS' MOTIVATION OF FIT BETWEEN STUDENT ABILITY 
AND THE LEVEL OF DIFFICULTY OF CAI PROGRAMS 



Ruth Miller and Robert D. Hess 



The purpose of this study was to identify some of the engaging 
characteristics of computer-assisted instruction (CAI), that is, 
the properties that arouse students' interest and maintain it over 
extended periods of time. Once having identified some of these en- 
gaging characteristics, we wished to determine (a) why engagement 
remains high on CAI programs , (b) whether it might be increased ex- 
perimentally by changing the level of difficulty of the lessons, 

(c) whether engagement is highest when lessons match the student's 
level of competence, and (d) whether engagement and disengagement 
scores are related. 

It has been shown that students in a computer-assisted arith- 
metic program develop positive attitudes toward the computer (Hass, 
Tenezakis , Smith, Brod, Spellman, Ingle, & Oppman, 1970). Students 
approach the computer room eagerly, and they appear to like working 
with the computer even when the instructional program continues 
throughout their elementary school years, as it does in the 
school from which the subjects in this experiment were drawn. 

Ruth Miller was a student in psychology at Stanford University 
when this report was prepared. She is now a graduate student in 
psycholinguistics at the University of California, Berkeley. Robert 
D. Hess is Lee L. Jacks Professor of Child Education at Stanford 
University and is director of the program on Teaching Students from 
Low- Income Areas at the Center. 
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The Math Drill-and-Practice Routine 

From the many types of programs grouped under the rubric of 
"computer-assisted instruction," we chose a math drill— and— practice 
routine devised by Patrick Suppes of the Stanford Institute for 
Mathematical Studies in the Social Sciences. This routine consists 
of lessons assembled from various texts and used as the teacher sees 
fit for extra drill and practice on skills in arithmetic. It is 
analogous to homework assignments and is constantly being revised. 

(For a general description of Suppes's work in this area, see Suppes, 
Jerman, & Brian, 1968.) The introduction and explanation of concepts 
is the responsibility of the classroom teacher. 

The Concept Block Approach 

From September 1966 to April 1970, the curriculum material in the 
math drill- and-practice routine for grades 1 through 6 was arranged 
sequentially in blocks to coincide approximately with the development 
of mathematical concepts introduced in several series of texts. 

(After April 1970 the concept block presentation was replaced by a 
"strands" approach.) There were 20 to 27 concept blocks for each grade 
level. The materials presented to the student for each of the seven- 
day blocks were as follows: 

Day 1 Pretest 

Days 2-5 Drill and review drill 

Day 6 Drill and review posttest 

Day 7 Posttest 

Each problem is printed for the student by the teletype, and the 
student types his answer in the appropriate place in each problem. If 
his answer is correct, the teletype prints the next problem. If his 
answer is incorrect, the teletype prints NO, TRY AGAIN, and presents 
the problem again. If there is a second error on the same problem, the 
teletype prints NO, THE ANSWER IS ..., and the problem is presented a 
third time. This step is included so that the student may have the 
experience of answering the problem correctly. If the student answers 
incorrectly for the third time, he is given the correct answer once more, 
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and the teletype prints out the next problem. The student is allowed 
from 10 to 40 seconds to respond, depending on the type of problem 
presented. If he takes more time than is allotted to type in his 
answer, the procedure just described is followed, except that the 
teletype prints TIME IS UP, TRY AGAIN in place of NO, TRY AGAIN. 

Under the concept block schema the level of difficulty of the 
first day of drill was determined by the student's performance on the 
pretest. The level of difficulty of each successive drill in the same 
concept block was determined by the student's performance on the pre- 
ceding day's drill. Thus, if the student's performance on a drill was 
80 percent or better, his next drill was one level of difficulty 
higher. A score of less than 60 percent brought him down a level for 
the next drill. If he scored between 60 percent and 80 percent he 
remained at the same level of difficulty for the next drill. (Descrip- 
tion taken from Suppes & Morningstar, 1969.) 

Distinguishing Characteristics of CAI 

Certain characteristics of the drill-and-practice lessons distin- 
guish them from other forms of instruction. First, CAI has special 
attention-getting attributes which stem simply from the fact that the 
material is presented by a machine. The carriage of the teletype is 

constantly in motion as the problems are typed and the answers ver- 

* > 

ified, and the machine buzzes like an electric typewriter. Thus, both 
sound and motion are important attention-getting attributes of CAI 
programs. 

The impersonal attributes of the machine's presentation further 
distinguish drill-and-practice programs from other forms of instruction. 
A machine is an unbiased grader of each performance. That is, the 
student's performance on anything other than his lesson is irrelevant 
to his grade on the lesson. This may relieve some of the anxiety 
inherent in an encounter between student and teacher during which the 
affective state of both persons may be changed. The student receives 
a score that reflects his performance, not the quality of the inter- 
action. 
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Other distinguishing features of CAI are individualization, small 
task-size, and feedback. "Individualization" means that the student 
progresses through the material at his owr rcr.e and at his own level 
of competence. "Task-size" refers to the amount of time spent at the 
computer; this time, 10 minutes a day, contrasts with a 45-minute time 
period required to present a similar lesson in the classroom. The 
pacing of the CAI presentation is rapid, and the flow of information is 
concentrated in a short period of time. "Feedback" is the information 
that the teletype gives the student about the correctness or incorrect- 
ness of his answer immediately after each problem. At the end of each 
lesson, the student learns from the computer what percent of the total 
problems he worked correctly. 

The students in schools that employ the CAI arithmetic routines 
work on these programs daily throughout elementary school. Our observa- 
tions of the students at work in the computer room confirmed that they 
do not lose interest in the lessons over the years. What keeps the 
students engaged in CAI over long periods of time? Why is there so 
little habituation to — and resulting disinterest in — this method of 
instruction? 

The Process of Engagement 

The process of engagement begins with an orientation to a stimulus 
source that arouses the person's (P's) interest. The stimulus source 
has a set of engaging characteristics. A sustained interest in the stimu 
lus source or in an activity dealing with the stimulus source may be 
indicated by a time line (see Fig. 1) that signifies the behavioral 
indicators of P's engagement at any moment. The initial point of P's 
engagement is I . The behaviors are supposedly motivated by the factors 
indicated above the time line. That is, some factors may play the 
most important role in initiating engagement, and others may be 
more important in sustaining engagement or in recalling interest at 
a later date or time. If engagement is not sustained, P's involve- 
ment ends, at point T g . 
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Fig. 1. Stages of engagement in CAI drill and practice. 
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Initial Engaging Features of CAI 

One important property of a set of engaging characteristics, we 
feel, is their ability to provoke curiosity. For a student working on 
a computer lesson for the first time, the engagement process would 
begin at a point when his curiosity was aroused and his attention was 
focused on the machine. According to Berlyne (1960) , curiosity is a 
state of heightened drive or arousal that increases with the degree o f 
novelty (change, surprise); the complexity (amount of variety or 
diversity in a stimulus pattern); the uncertainty (of categorization) 
of the stimulus; and the degree of conflict among the possible responses 
to the stimulus. He uses the term ’ ’perceptual curiosity " to refer to 
"states of high arousal that can be relieved by specific exploration 
and in which, therefore, specific exploratory responses are likely to 
occur [p. 195].’’ In themselves these exploratory responses reduce 
novelty and complexity as habituation to the stimulus occurs. Uncer- 
tainty and conflict may be reduced by descriptive or explanatory 
naming and/or through the learning of a new response. 

We hypothesized that at the initial point in the engagement pro- 
cess it is the use of a mechanical device to present the lessons that 
arouses the students' curiosity. The teletype is a novel and surprising 
instrument. It looks somewhat like a typewriter, but it can type 
without the student hitting the keys. It appears complex in design, and 
the students must learn the correct way of responding in order to work 
their lessons. It is imperative that the teletype have some familiar 
characteristics because, Berlyne asserts, if a stimulus is completely 
novel, it will arouse fear and withdrawal rather than curiosity and a 
desire to experiment. 

The curiosity and resulting exploratory behavior induced by the 
first days with the teletype may be supplemented by a "manipulative 
drive" such as that proposed by Harlow and his associates (Harlow, 1953; 
Harlow, Harlow, & Meyer, 1950). This drive was postulated as a result 
of their experiments with rhesus monkeys, in which the monkeys repeatedly 
solved a mechanical problem even when it led to no further consequences 
or rewards. It does not seem likely that novelty is the characteristic 
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of the stimulus that evokes this repeated behavior. Rather, the 
results could be interpreted as showing a desire to play or "tinker" 
or, more generally, as Zimbardo and Miller (1958) express it, "to 
effect a stimulus change in the environment." 

Thus we postulated that CAI is initially successful in capturing 
students' attention because of its curiosity-provoking aspects and 
because the program is presented by a teletype whose keys must be 
punched or manipulated. The teletype is novel; it comprises surprising 
elements; it is complex; and the students must learn about and catego- 
rize all these elements to be able to respond to it. This covers the 
initial engaging features of CAI. Our experiment, however, dealt with 
the features of CAI that continually engage the students after they 
have become Accustomed to the use of the machine. 

Features of CAI that Maintain Engagement 

We use the term "feedback" in two senses. In the first (FI), 
feedback refers to the specific response the computer gives the student 
immediately after each answer. We regard this immediate feedback as one 
of the most important engaging characteristics of computer-assisted in- 
struction. In the second sense (FII), feedback refers to an inner 
response during which the student evaluates his performance, or rates 
his competence, on the basis of FI, that is, the computer's evaluation 
of his work on the problems . 

White (1959) introduced the concept of "effectance motivation," 
which, he said, "aims for the feeling of efficacy, not for the vitally 
important learnings which come as its consequence." The behavior 
influenced by effectance motivation involves "focal attention to some 
object — the fixing of some aspect of the stimulus field so that it 
becomes relatively constant — and it also involves the focalizing of 
action upon this object in order to effect some change in the object 
(p. 322)." White said: 

The urge toward competence is inferred specifically from 
behavior that shows a lasting focalization and that lias the 

characteristic of exploration and experimentation, kind 
of variation within the focus. When this particular sort of 
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activity is aroused in the nervous system, effectance motiva- 
tion is being aroused, for it is characteristic of this 
particular sort of activity that it is selective, directed, 
and persistent, and that instrumental acts will be learned 
for the sole reward of engaging in it [p. 323]. 

We hypothesized that as the student works on the drill-and- 
practice programs he is engaged in a form of "competence testing." 

In computer-assisted instruction the student learns how to respond to 
a novel and complex stimulus, and in the process produces changes in 
the stimulus. New responsibilities for response are offered in the form 
of a continual flow of math problems to be solved. As the student re- 
ceives feedback about his performance (FI) , his successfulness in the 
transaction, or his competence, is confirmed (FII) . 

We also hypothesized that the importance of FI to the student 
depends on the level of difficulty of the lesson he receives. If 
the level of difficulty of the lesson is slightly above the student's 
proficiency in arithmetic, then he can use the information provided by 
FI to test his competence in solving the problems. A correct answer is 
gratifying because he has met the challenge to his competence. If the 
lesson is too easy, his competence at this level has already been 
proven. If the lesson is too difficult, eventually he will reject it. 

We predicted that engagement in the programs would be sustained 
only if the level of difficulty of the lessons was suited to the student's 
highest level of accomplishment. We call this his "level of competence." 
Because the students do receive lessons that give them practice in 
solving problems matched to their learning pace in the classroom, we think 
this programming is another important engaging feature of CAI. 

If our hypotheses are correct, one should be able to demonstrate 
losses of engagement experimentally by varying the levels of difficulty 
of the CAI lessons presented to a student. A student's engagement in 
an arithmetic lesson that is either too easy or too difficult for him 
in terms of what he has learned in the classroom should be less than 
his engagement in a lesson that is matched to his level of competence. 

We also wished to test another hypothesis: that one can experimen- 
tally demonstrate increased engagement by changing the level of difficulty 
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of a lesson when the student finds the problems too hard to solve. 

To do so we made a series of ''clues" available to the students. 

When they were confronted with a problem that they had not learned 
how to solve, they could ask for one of the clues to its solution. 

As noted earlier , some of the most important properties of CAI 
for attracting attention are novelty, incongruity, surprise, and 
change. Another of these properties, conceptual conflict, we felt 
could not be directly examined in the design of the math drill-and- 
practice routines. Conceptual conflict, or a discrepancy between what 
one knows and what one wants to know, is effective in mobilizing 
attention (engaging a student) while one is seeking information to 
resolve the discrepancy (gap in knowledge) or solve the problem 
(Berlyne 1960) . Berlyn uses the concept "epistemic curiosity" to 
describe the motivational state aroused by conceptual conflict 
"Epistemic behavior" is activity that is directed toward reducing 
epistemic curiosity through acquiring the knowledge needed to fill 
the gap. The discrepancy between what one knows and what one wants 
to know must not be too large, however, or the conflict will produce 
frustration and disinterest. 

In reference to the hypothesis just mentioned, if a student 
encounters a problem in a CAI arithmetic lesson that is too difficult 
for him to solve, yet may be barely beyond his ability, both concep- 
tual conflict and epistemic curiosity will be aroused. If he can ask 
for a clue to the solution of the problem in order to fill the gap in 
his knowledge, his epistemic behavior will be rewarded, and his engage- 
ment in the set of problems that were once too difficult should 
increase. 

Method 

Subjects in the experiment were students from the fourth grade at 
Brentwood Elementary School in East Palo Alto, California. All had 
been in the CAI math drill-and-practice routine since first grade. 

Only nine out of an estimated forty subjects were included in the 
final study because of faulty videotape equipment used in filming the 
experiment. 

i 

The setting for the experiment was two rooms at Stanford Research 
Institute, Menlo Park, California. The students received the 
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experimental lessons from a PDP— 10 teletype similar to the one they 
used every day at Brentwood Elementary School. Differences between 
the two were explained to them before the experiment! (1) they had 
to press the "return" key after every answer; (2) the carriage 
returned to a space below the problem for the answer, rather than 
to a blank within the problem itself; and (3) the computer did not 
"sign off" at the end of each lesson. The students had no difficulty 
with the first two differences, but the third caused a great deal of 
difficulty because several students did not call the experimenter 
after they completed the lesson, but instead worked it a second time 
(and in some cases a third) before the experimenter could give them 

a new lesson. 

The students were brought to SRI in private cars. After they 
met the experimenter at the entrance of the building, they were 
given a short tour of the building by the experimenter or by Dean 
Brown, a researcher at SRI. This informal tour gave them an 
opportunity to become acquainted with their surroundings. Then 
they were taken to the floor that houses the computer hardware for 
SRI. The students were allowed to ask questions about the hardware, 
to work with any of the programs in Mr. Brown's repertoire, and to 
type on the card-punch machine. As the students were experimenting 
with these machines, they were invited, two at a time, to go with 
the experimenter to try out some new arithmetic programs. 

Each student was taken to an office and seated before a PDP-10 
teletype. The videotaping equipment in one office was on a metal 
cart about 4-1/2 feet high. The camera was on a tripod about 5 feet 
high. The photographer sat behind the cart out of the view of the 
student. The experimenter explained that a movie was going to be 
made of the students while they were working, but encouraged 
them to forget about the camera and to pay attention to the les- 
sons because there would be a time limit for their answers just 
as at school. The students were not allowed to look through the 
camera because the cameras were pre-focused on the teletypes and 
only a short time was allotted for filming each child. The camera 
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in the other office was set up outside the doorway to the office. The 
photographer sat in the doorway out of the sight of the student. 

The students generalized from one teletype to the next when the 
le 8 sons were similar to those they used at their school. They quickly 
became absorbed in their work and seemed .to forget the presence of the 
camera and the cameraman, with one notable exception. 

The experimenter predetermined the order in which the students 
received the five lessons. The orders were not random, because lesson 
always followed (though not immediately) the presentation of lesson 
H^. Because of the small number of subjects in the experiment, the 
experimenter did not attempt a random ordering of the presentation of 
the lessons. Care was taken, however, to demonstrate the presence 
or absence of position effects in the results by attempting to give 
some of the subjects an easy lesson after they had completed a 
difficult one. Other children received the lessons in the order 
easy-to-hard. (See Table 1.) 

Description of Experimental Lessons 

The experimenter designed five arithmetic lessons to be presented 
to the subjects by a teletype similar to the one they had been using 
daily at school. The code names of the experimental lessons were E^, 
Eg, E^, H^, and Hg. The contents of. the first four of the lessons were 
directly modeled after the Suppes routine. 

The first set of problems, E^, were addition problems with the 
format : 

1. a + b = , a 

+ b or; 



2. a + be = , be + a = or; 

3. a + = c, + b = c. 

It was not necessary to carry any sums across columns. This lesson 
would be ordered at grade 2.0 difficulty in the Suppes routine. There 
was a 10-second time limit for answering the problems. 
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TABLE 1 



Summary of Student 


Performance 


and Level of 


Engagement 










Lessons 








E i 


E 2 


E 3 


H i 


H 2 


Mark * 

Percent correct 


100% 


100% a 


100% 


50% 


80% 


No. of problems worked 


10 


20 


20 


10 


10 


Time in minutes 


1.0 


2.5 


3.0 


4.0 


2.5 


Engagement points 


29 


56 


98 


110 


64 


Disengagement points 


0 


16 


6 


24 


12 


Order of presentation 


5th 


4th 


1st 


2nd 


3rd 


George 


Percent correct 


ioo% a 


20% 


20% 


20% 


20% 


No. of problems worked 


10 


10 


10 


10 


10 


Time in minutes 


1.5 


4.0 


4.5 


4.5 


5.5 


Engagement points 


30 


101 


114 


94 


114 


Disengagement points 


5 


21 


26 


30 


43 


Order of presentation 


1st 


3rd 


4th 


2nd 


5th 


Sally 


Percent correct 


100% 


100% 


80% a 


100% 


90% 


No. of problems worked 


20 


20 


10 


10 


10 


Time in minutes 


1.0 


1.5 


1.5 


1.5 


2.5 


Engagement points 


27 


44 


37 


41 


70 


Disengagement points 


0 


1 


1 


5 


7 


Order of presentation 


1st 


2nd 


3rd 


4th 


5th 


David 

Percent correct 


95% 


100% a 


90% 


70% 


90% 


No. of problems worked 




20 (30) 


10 


10 


10 


Time in minutes 


3.5 


5.5 


2.5 


6.0 


4.0 


Engagement 


98 


147 


61 


135 


97 


Disengagement 


9 


22 


10 


40 


22 


Order of presentation 


3rd 


1st 


5th 


2nd 


4th 



13 



TABLE 1 continued 



Lessons 





E i 


E 2 


E 3 


H i 


«2 


Mabel 


Percent correct 


95% 


100% a 


95% 


0% 


20% 


No. of problems worked 


20 


20 


20 


8 


10 


Time in minutes 


2.5 


4.0 


2.0 


5.0 


6.0 


Engegement 


69 


105 


57 


126 


124 


Disengagement 


8 


10 


4 


37 


44 


Order of presentation 


3rd 


5th 


3rd 


1st 


2nd 


Joseph 


Percent correct 


100% 


95% 


70% 


60% 


80% 


No. of problems worked 


20 


20 


10 


10 


20 


Time in minutes 


3.5 


4.0 


4.0 


2.5 


2.5 


Engagement 


87 


92 


75 


66 


156 


Disengagement 


12 


4 


17 


1 


10 


Order of presentation 


1st 


2nd 


3rd 


4 th 


5th 


Amy 


Percent correct 


90% 


40% 


30% a 


100% 




No. of problems worked 


20 


10 


10 


20 




Time in minutes 


2.5 


2.5 


3.0 


3.0 


1.5 


Engagement 


61 


55 


78 


72 


33 


Disengagement 


5 


21 


13 


16 


17 


Order of presentation 


2nd 


3rd 


1st 


4 th 


5th 


Alice 


Percent correct 


100% 


90% a 


60% 


10% 


100% 


No. of problems worked 


10 


10 


10 


10 


10 


Time in minutes 


2.0 


1.0 


4. 5 


3.0 


2.5 


Engagement 


63 


31 


131 


63 


78 


Disengagement 


3 


1 


35 


39 


11 


Order of presentation 


4th 


5 th 


3rd 


1st 


2nd 
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TABLE 1 continued 






Lessons 








E 2 


E 3 


H 1 


H 2 


Jan 












Percent correct 


90% 


80% a 


30% 


100% 


100% 


No. of problems worked 


20 


10 


10 


10 


10 


Time in minutes 


5.0 


3.0 


2.5 


3.0 


3.0 


Engagement 


186 


110 


67 


103 


62 


Disengagement 


15 


38 


33 


5 


2 


Order of presentation 


3rd 


2nd 


1st 


4th 


5th 



* Names have been changed to pseudonyms. 
a Lesson at subject's level of competence. 



The second set of problems, E 2> were multiplication and division prob- 
lems with the format: 

1. a X b = , aX = c, Xb = c or; 

2. a / b = , a / = c, / b = c. 

All the products ranged from 0 to 24. This lesson would be ordered at grade 
2.8 difficulty in the Suppes routine. There was a 10-second time limit for 

answering the problems. 

The third set of problems, E^, were multiplication and division problems 
with the same format as the problems in E 2 : 

1. a X b = , a X = c, X b = c or; 

2. a / b - , a / = c, / b = c. 

All the products ranged from 27 to 81. This lesson would be ordered at 
grade 3.3 difficulty in the Suppes routine. There was a 10-second time 
limit for answering the problems. 
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The fourth lesson, H^, was a set of problems demonstrating the use 
of the commutative, associative, and distributive laws of arithmetic. 

Here are some examples of the use of the laws : 

1. Commutative Law 

a X b * b X a or a + b = b + c 

2. Associative Law 

(a X b) X c = a X (b X c) or (a + b) + c = a + (b + c) 

3. Distributive Law 

a X (b + c) = (a X b) + (a X c) 

The problems in the lesson involved filling in a missing variable on the 
left side of This lesson would be ordered at Grade 3.8 difficulty 

in the Suppes routine. There was a 10-second time limit for answering 
the problems. 

The fifth lesson, H 2 , consisted of the same problems as H^. In 
H 2 » however, if a subject did not know how to solve a problem, he could 
type "H" in place of an answer. Then he received an explanation of the 
law and two examples of the way the law is used. After receiving this 
"help," the student was given another chance to work the problem. This 
program is not included in the Suppes routine. There was no time limit 
for answering the problems in H 2> 

There were 20 problems in each lesson. After the first ten prob- 
lems, the computer typed: "Are you ready to continue these problems? 

Please respond by typing a YES or NO." Making it possible for a subject 
to terminate a lesson after the first ten problems gave him a measure 
of control over the amount of time spent on each lesson.-. The experimenter 
added this option after she found that the subjects were. spending more 
time on the experimental lessons than they spent answering their daily 
CAI lessons. The subject's YES or NO response to this question was not 
used as a variable to rate his level of engagement in the program 
because the experiment did rot include a method of determining the 
exact reason why he made one response rather than the other. 

According to the Suppes curriculum guide, the first four experimen- 
tal lessons were ranked in order of increasing difficulty: E^, E 2 , E^, H^. 

The experimenter determined which lesson was at each subject's level of 
accomplishment by matching the grade level at which he was working in the 
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CAI program to the grade level of one of the experimental lessons. 

At the time the experiment was designed, none of the subjects had been 
introduced to the commutative, associative, or distributive laws (CAD 
laws). For this reason, only lessons E^, E 2 , or E^ could be designated 
as a program at a subject's level of competence. 



Level of Competence 
E. 



Grade Level 

2.0 - 2.7 
2.8 - 3.0 

3.1 - 3.7 



Number of Subjects 
at this Level 



Because none of the subjects had been introduced to the CAD laws, 
it was expected that H 1 would be the most dfficult program for all of 
them. Therefore, all but one of the subjects were given at least one 
program that was less difficult than the program at their level of 
competence, and all of the subjects were given at least one program that 
was more difficult than the program at their level of competence. 

Between the time the experiment was designed and the time it was executed, 
however, an important methodological difficulty arose. Many of the 
students in the drill-and-practice routine were introduced to the use 
of the CAD laws when a "strands" rather than a "concept block" form of 
programming was used to move the students through the lesson plans . 

During the course of the experiment the students were receiving lessons 
at random in order to place them at their proper level of competence 
in the strands program; the difficulty of the lessons that the students 
were receiving was not recorded, nor was the students' level of com- 
petence recorded by grade level (2.8, for example). Thus it was im- 
possible for the experimenter to rank lesson H 1 as the most difficult 
of the experimental lessons for every student. The final ranking of 
the lessons Ep and E 3 in order of increasing difficulty had to be 
based on the school records prior to the experiment; and the rank of 
the difficulty of H^had to be based on the subjects' performance score 
on this lesson during the experiment. Therefore, it was possible to 
rank the lessons differently for a student if he had been introduced 
to the CAD laws before the experiment. For example, if a student 
scored 100 percent on lesson during the experiment, it would be 
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assumed that she had been introduced to the use of the CAD laws before 
the experiment. In such a case, the order of difficulty would be adjust- 
ed accordingly. For one student the lessons were ordered by increasing 
difficulty E^, H^, E 2 , E^; whereas for the rest of the subjects the order 
by increasing difficulty remained E^., E 2 * Eg* H^. 

We hypothesized (a) that the rating of the subject's engagement in 
the lessons E^, E 2 , Eg, and would be highest for the lesson that 
matched his level of competence; and (b) that there would be observable 
losses of engagement (compared with this rating) when the lessons were 
less difficult or more difficult than his levei of competence. We also 
hypothesized (3) that if the subjects used the clues in Hg, their engage- 
ment in this lesson would be greater than their engagement in H^. 

Development of the Observation Instrument 

In order to rate the extent to which the subj ects were engaged in 
the experimental lessons, the experimenter developed an observation 
instrument to be used while viewing the videotapes of the experiment. 

Behaviors observed on the tapes were divided into five modes of expres- 
sion: postural , visual, mobile , gestural , and facial. Behaviors that 
indicated engagement and behaviors that indicated disengagement were 
defined. Behaviors that were evidence of engagement indicated the 
student's attentiveness — his curiosity, interest, involvement, and 
persistence in the activity. Behaviors that evidenced disengagement 
were inattentiveness to the activity, disinterest, and lack of curiosity 
about the activity. 

Engagement in the postural mode was expressed by alertness. In the 
visual mode engagement was reflected in the student's focus of atten- 
tion; if he was engaged, his eyes were on the stimulus source. The 
mobile behaviors expressing engagement had to do with the student's 
active proximity seeking, that is, his movements bringing him into 
closer contact with the stimulus source. Gestural and facial expres- 
sions of engagement are largely idiosyncratic. In the measurement we 
included gestural and facial expressions that are manifestations of 
problem solving. There may also be an effective mode of expression, 
which includes behaviors indicating a mood of satisfaction. 
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Disengagement in the postural mode is expressed by fatigue or bore- 
dom. Again it involves focus of attention in the visual mode; if the 
student is disengaged, his eyes will have wandered away from the stimulus 
source. Mobile behaviors expressing disengagement have to do with the 
child's active withdrawal from the stimulus source. Gestural and facial 
expressions of disengagement — which like such expressions of engagement 
are idiosyncratic — may include restless behaviors, protest, and 
dejection. There may also be an affective mode of expression of dis- 
engagement which includes behaviors indicating a mood of dissatisfaction. 

Categories of Engagement and Disengagement 

A description of the engaged and disengaged behaviors that were 
used in rating the students’ behavior is as follows: 



1. Postural Behavior 



A. 



B. 



C. 



D. 



"(++)" Child, leaning, forward to examine, paper . 

"(+)" Child's posture is "normal," i.e., he is sitting with 
weight supported by lower half of body. Spine may be 
straight or relaxed. 

Child is leaning on arm of chair. Weight is not evenly 
distributed. A "slumping" posture. 

"( — )" Child is leaning back in his chair (hips forward) in a 
position he would have to alter with a whole body move- 
ment, rather than a partial one, if he returned to (+) 
position. 

Or, the child is leaning on the teletype, usually with 
his head resting oh the machine. 



2. Visual Behavior 



A. "Eyes on paper" 

Child is looking at the paper or the keyboard. 

B. "Vacillation" 

Child's gaze shifts away from paper and keyboard and re- 
turns to them (or vice-versa) within the five-second 
observation period, i.e., child is both "looking away" 
and"looking at paper" in that five-second period. 
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C. "Looking away" 

Child is looking at anything other than the paper or 
keyboard. This may include looking at some other part 
of the machine (sides), out the window, etc. 

3. Mobile Behavior 

A. "Pulls body or chair closer to teletype." 

Child leans forward to inspect paper; straightens posture 
noticeably in order to be able to see paper, hit the keys 
faster, etc. ; or pulls his chair closer to the machine for 
any of the reasons above. This behavior may signify 
renewed or increased interest in a set of problems. 

B. "Touches paper." 

Child straightens paper; lifts aaper to see the problems 
he has completed; or lifts paper to read a long paragraph 
the machine has typed. 

C . "Turns around . " 

Child turns his body around in the chair so that his torso 
is at an angle to the machine. 

D. "Pulls back." 

Child lets his weight fall back into a posture defined as 

(-). 

4. Physical Manifestation of Problem-Solving 

A. "Reading silently." 

B. "Talks to .self." 

C. "Counts on fingers." 

D. "Finger to mouth." 

E. "Surprise." 

Child indicates he receives unexpected feedback. 

5. Disengagement 

A. "Actively resistent." 

Child rejects program, feedback, or difficulty of problem. 

B. "Withdraws." 

Listless. Child is no longer interested in reading problems. 

C. "Restless." 

Child fidgets, flounces, etc. 
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D. "Talking to someone." 

6. Affective Behavior 

A. "Satisfaction." 

Child derives pleasure from producing the correct response 
or working on the set of problems. 

B. "Dissatisfaction." 

Child indicates displeasure with the program or his answer. 



Tallying Observations 



Engagement 



Disengagement 



1. 


(++) 


2 points 


2. 


(+) 


1 


It 


3. 


Eyes on paper 


1 


II 


4. 


Pulls body, chair 
close to teletype 


1 


II 


5. 


Touches paper 


1 


II 


6. 


Reads silently 


1 


II 


7. 


Talks to self 


1 


II 


8. 


Counts fingers 


1 


II 


9. 


Finger to mouth, 
scratches head 


1 


II 


10. 


Surprise 


1 


II 



1 . (-> 

2 . (-) 

3. Vacillation 

4 . Looks away 

5 . Turns around 

6. Pulls Back 

7. Actively resistant 

8. Withdraws 

9. Restless 

10. Talking to 
someone 



2 points 
1 " 

1 " 

1 " 

1 " 

1 " 

1 " 

1 " 

1 " 

1 " 



The raters viewed the videotape of three subjects working on the ex- 
perimental lessons. The behavior was recorded every five seconds. After 
the observation was completed, the raters tallied the scores for engage- 
ment and disengagement per minute and graphed these totals for each 
lesson. Figure 2 is an example of one of the graphs. No affective 
behavior was observed. 
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Fig. 2. 



Engagement and disengagement scores . 



Indices and Variables 

Data were analyzed from several different perspectives. From the 
raw scores of engagement and disengagement per lesson for each child, 
the following set of variables and indices was derived: 

1. E/T Engagement /Time 

Engagement points per minute of time spent on the lesson 
tallied for each of the subject's five lessons. 

2. D/T Disengagement /Time 

Disengagement points per minute of time spent on the lesson 
tallied for each of the subject's five lessons. 




4 



26 



22 



3. E/D Ratio of Engagement to Disengagement 

Derived by dividing the number of disengagement points re- 
ceived for one lesson into the number of engagement points 
received for the same lesson.^ - 

4. Rank of Lesson's "Engagingness" 

Lesson ranked 1 (most engaging) to 5 (least engaging) for each 
subject, based on rates E/D. 

5. Predicted Level of Difficulty Adjusted to Subject's Level of 
Competence 

Origin is the lesson at each subject's level of competence as 
determined by the school records. Lessons decrease in difficulty: 
-1, -2, -3... and increase in difficulty: +1, +2, +3.... 

In order to graph the relationship of E/T, D/T, and E/D to the subject's 
level of competence as shown on an experimental lesson, the experimenter 
ranked the lessons for each subject, from least difficult to most difficult. 
The lessons were put into matrix form with the lesson that matched each 
subject's level of competence at the "0" point. A lesson that was less 
difficult by one rank was put in the "-1" column; a lesson that was more 
difficult by one rank was put in the "+1" column, etc. (See Tables 3, 

4, 5.) 

6. "Actual" Level of Difficulty 

Determined by the subjects' percent correct scores on any 
lesson. 100 percent correct compares to lessons that are (-1) 
level of difficulty; 99-80 percent correct would include the 
lesson at the subject's level of competence, the "0" point; 

79-50 percent correct compares to lessons that are (+1) level 
of difficulty; 49-0 percent correct compares to lessons that 
are (+2) level of difficulty. The summary of the analysis 
of the results for each student is shown in Tables 6 and 7. 



There was a total of 45 lessons for the 9 subjects. Two of thp sub- 
jects did not receive a score for disengagement on one of their lessons. 
This means that 2 lessons out of 45 had D = 0. It was possible to compute 
D/T when D = 0, but it was not possible to compute E/D when D = 0. 

In order to compute E/D, the experimenter made D = 1 for these two lessons, 
justifying this manipulation for the following reasons: since D varies from 
0 to 44, the difference of the addition of one point is not as great as it 
would be if the variance were small, e.g., 0 to 5; furthermore, the two 
lessons where no disengagement score was recorded were observed for only 
one minute, and because the average time spent on each lesson was 3.1 
minutes, the experimenter believed that one minute was not, perhaps, an 
adequate amount of time for a subject to become disengaged in an easy pro- 
gram. All of the subjects that were observed for 1.5 minutes received at 
least D » 1, and therefore the experimenter believed that making D ■ 1 
in 2 cases did not constitute a misleading representation of the data. 
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TABLE 2 



Indices of Engagement and Disengagement for Individual Subjects 



Mark 



George 



Sally 



David 



Joseph 




E/T 



29 

22 

33 

27 

20 

25 

25 

21 

27 

29 

25 

27 

27 

27 

25 

22 



25 

13 



19 



D/T 


E/D 


0 


29 :l a 


6 


3.6 


2 


16.5:1 


6 


4.6:1 


3.3 


6:1 


5.5 


5:1 


5 


4.4:1 


7 


3:1 


0 


27 :l a 


0.7 


44:1 


0.7 


37:1 


3.3 


8:1 


2.4 


11:1 


4 


67:1 


4 


6:1 


6 


3.4:1 


3.5 


7:1 


1 


23:1 


4.2 


4.4:1 


0.4 


66:1 
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TABLE 2, Continued 





L 


E/T 


D/T 


E/D 


Mabel 


E i 


28 


3.1 


8.6:1 




E 2 


25 


2.5 


10:1 




E 3 


29 


2 


14:1 




H 1 


25 


7.4 


3:1 


Amy 


E 1 


24 


2 


12:1 




E 2 


22 


8.4 


2.6:1 




E 3 


26 


4.3 


6:1 




H 1 


24 


5.3 


4.5:1 


Alice 


E 

1 


31 


1.5 


21:1 




E 2 


31 


1 


31:1 




E 3 


29 


7.8 


3.7:1 




H 1 


21 


13 


1.6:1 


Jan 


E 1 


37 


3 


12:1 




E 2 


37 


1.3 


2.9:1 




E 3 


27 


13 


2:1 




H 1 


21 


.67 


21:1 



D = 1 for computation of ratio. 
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Distribution of "E/lV by Ranking Predicted Level of Difficulty of Lessons 
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TABLE 6 



Ratio of Engagement to Disengagement by Actual 
Level of Difficulty of CAI Lessons 





100% 


99* 


-80% 


79- 


-50% 


49- 


-0% 


Student 


L 


E/D 


L 


E/D 


L 


E/D 


L 


E/D 


Mark 


E i 


29 






H i 


4.6 








E 3 


16.5 
















E 2 


3.5 






• 








George 


E 1 


6.0 






E 2 


5.0 
















E 3 


4.4 
















% 


3.0 






Sally 


E 2 


44.0 


E 3 


37.0 




* 








E 1 


27.0 
















H 1 


8.0 














David 


E 2 


6.7 


E 1 


11.0 


H 1 


3.4 












E 3 


6.0 










Mabel 


E 2 


10.0 


E 3 


14.0 






H i 


3.0 








E ! 


8.6 










Joseph 


E 1 


7.0 


E 2 


23.0 


H 1 


66.0 
















E 3 


4.4 






Amy 


H 1 


4.5 


E 1 


12.0 






E 3 


6.0 
















E 2 


2.6 


Alice 


E 1 


21.0 


E 2 


31.0 


E 3 


3.7 


H 1 


1.6 


Jon 


H 1 


21.0 


E 1 


12.0 






E 3 


2.0 








E 2 


2.9 










Average E/D 




15.7 




15.7 




11.8 




3.0 



29 



TABLE 7 

The Average Levels of Engagement and Disengagement per Minute 
in Relation to Actual Level of Difficulty 



100% 




99- 


■88% 


VO 

K 


■50% 


49-0% 




E/T 


D/T 


E/T 


D/T 


E/T 


D/T 


E/T 


D/T 


29 


30 


27 


2.4 


27 


6 


25 


7.4 


33 


2 


25 


4 


25 


5 


26 


4.3 


22 


6 


29 


2.0 


25 


5.5 


22 


8.4 


20 


3.3 


28 


3.1 


21 


7 


21 


13 


27 


4 


13 


1.0 


22 


6 


27 




29 


10 


24 


2.0 


26 


.4 






27 


3.5 


31 


1.0 


19 


4.2 






27 


5.3 


37 


3.0 


29 


7.8 






25 


1.5 














25 


.7 














24 












• 




21 
















21 
















Av. 26 


2.4 


27.6 


2 


24 


5.2 


24 


9.2 



Results 

Our predictions were that: (a) E/T would be highest (and D/T would 
be lowest) for the lesson that best matched the subject's level of accom- 
plishment in arithmetic; (b) E/T would be lower (and D/T would be higher) 
for lessons that were either less difficult or more difficult than this 
base; and (c) E/T would be higher (and D/T would be lower) on lesson Hg 
than on lesson if the subject made use of the hints available in 
to decrease the level of difficulty of the problems. It was not possible 
to analyze the results of (c) because the subjects did not understand the 

directions about when to ask for help. 

If these predictions (a) and (b) were correct, the graph would appear 

as in Figure 3. 
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Fig. 3. Hypothetical relationship between level of 
difficulty, engagement, and disengagement. 



The results are represented graphically because of the small size 
of the research group. The changes in engagement and disengagement over 
time as the lessons become either more difficult or less difficult is an 
important aid to understanding the engagement process as we have described 
it. 

The ratio of engagement to disengagement is a useful way to rep- 
resent the data because it shows the ratio of the number of points 
received for "engaged" behavior to every point received for "disengaged" 
behavior by the same subject in the same amount of time. For example, 
if the ratio of engagement to disengagement is 29:1, the subject was 
very much engaged in the lesson; he received only one point for dis- 
engaged behavior for every 29 points for engaged behavior. If, however, 
the ratio is 3:1, the subject was not very much engaged in the lesson; 
he received one point for disengaged behavior for every three points for 
engaged behavior. 
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Findings 

The data did not confirm our predictions when the difficulty of the 
lessons was below the subject's level of competence. The relationship of 
E/T to the predicted level of difficulty shows that engagement recorded 
for the easy lessons was a few points higher than engagement recorded at 
the level of competence. E/T dropped only at the lesson rated two ranks 
more difficult than the lesson at the level of competence, and it dropped 
only about seven points. (See Figure 4.) 

The relationship of D/T to the level of predicted difficulty shows 
that the disengagement recorded for the easy lessons was low, about the 
same as that recorded for the lesson at the level of competence. As the 
lessons became more difficult, the amount of recorded disengagement rose by 
a few points and remained there. (See Figure 4.) The relationship of the 
-atio E/D to the level of predicted difficulty appears random, possibly 
because of the unusual results within the small sample. 




Fig. 4. Relationship of engagement to the predicted level of diffi- 
culty of the lessons. (The predicted difficulty was determined by sub- 
jects' level of competence. The difficulty of E]_, E 2 , and E 3 for each 
subject was taken from school records; the difficulty of Hi was determined 
by performance on an experimental lesson.) 
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When the level of difficulty of the lessons is represented accord- 
ing to the subjects* performance on them during the experiment (see 
Figure 5), there is a bit more substantial support for our predictions 
on the positive side of the point of origin. The relationship of E/T 
to the actual level of difficulty shows that E/T was high and remained 
almost constant until the subject missed more than 20 percent of the 
problems per lesson. After the subject missed more than 20 percent of 
the problems, E/T dropped slightly and again remained constant, even 
when the subject missed more than 50 percent of the problems. 

The relationship of D/T to the actual level of difficulty of the 
lessons showed that D/T was low until the subject began to miss more 
than 20 percent of the problems. As the lessons became more and more 
difficult and the subject began to miss more than 50 percent of the 
problems, more disengagement was recorded, and D/T rose. 




Fig. 5. Relationship between the indices of engagement and actual 
level of difficulty in CAI. (The actual scores of difficulty of the 
lessons was determined by percent correct scores.) 
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Again the data show that the subjects 1 engagement did not drop as 
the lessons became less difficult than the lesson at their level of com- 
petence. Engagement is high and disengagement is low on the lessons that 
were easy for them to do. On the other hand, the ratio of E/D began to 
drop quickly as the subjects missed more than 20 percent of the problems 
or as the lessons became too difficult for them. 

In Figure 6 it can be seen that our subjects were more interested in 
the easy programs (the lessons they did well on) than, in the difficult 
ones (the lessons they did poorly on). The fl engagingness fl of the lesson 
for each subject was determined by a comparison of the ratio E/D for all 
the lessons. 
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Fig. 6. Relationship of the ease of the lesson to its motivating 
•properties . 
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Why did our subjects show the most interest in the easy problems? 
According to our hypotheses, these lessons should not have provided an 
adequate challenge to their competence. Actually, our results indicate 
that there is a wider margin of tolerance for easy lessons than for 
difficult ones. 

If this is so, the graph of the relationship of E/D to the level of 
difficulty of the lesson should look something like Figure 7. 




Fig. 7. Hypothetical relationship between engagement 
and ease of lesson. 



We predict that there is a point at which the lesson will be so easy 
that it is disengaging, but that the point is much further away from 
the child's level of competence than we had originally predicted. Why 
might this be so, and why might we have obtained the results that we 
did in the experiment? 




*■ 



39 



35 



The questions we wished to answer were: (1) Why does engagement 
remain high throughout the lessons? (2) Where is the point of maximum 
engagement? (3) Are engagement scores and disengagement scores related? 

The teletype is a complex instrument. It cannot accept a mis-typed 
response any more than it can accept an incorrect answer to a problem. 
Students then probably have to maintain a high level of engagement in 
the programs simply to avoid making typing errors. Another possible 
reason for the high engagement scores is that standards in the experimental 
environment may have appeared more rigorous than those in the computer 
classroom. As a result, students may have been highly engaged in the 
programs at which they could excel because they needed more assurance of 
their competence. It is also possible that because no competing stimuli 
were present during the experiment, the subjects were not tempted to play 
with the easy programs as they do in the computer classroom where many 
other students are working at the same time. 

Possibly we did not place enough importance on the engaging properties 
of the machine itself. The results of the experiment show that engagement 
remained high for the length of the experiment — for some of the subjects, 
one hour. The data may suggest that students do not become habituated 
to the machine, but remain continually interested in seeing what the 
machine will type next. Thus, the element of surprise may never be 
lost. 

We found no answer to the second question: Where is the maximum 

point of engagement? According to our predictions, a lesson at the 
student's level of competence, or one slightly above that level, would 
be the most engaging. Instead, the students found the easy lessons 
just as engaging — or more engaging — than those at their level of com- 
petence. If the graph of the relationship of E/D to the level of 
difficulty of the lesson looks like Figure 7, it may be that our data 
fit only a section of this curve. It should be possible to provide a 
lesson so easy that engagement drops, as well as a lesson so difficult 
that, in the complete absence of reward, engagement drops. 

In answer to the third question, we find that E/T and D/T vary 
independently. We can, therefore, regard engagement and disengagement 
as independent measures. Consequently the amount of engaged behavior 
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may remain high while the amount of disengaged behavior is rising. Thus 
we might predict that a rise in disengaged behavior will be a more sen- 
sitive indicator of loss of engagement than an actual drop in recorded 
engaged behavior. This is to say, the student may still be attentive 
to the lesson although he is beginning to be restless and to fidget. 

Unless something occurs to recapture his attention, engaged behavior may 
become less frequent as disengaged behavior becomes more frequent. This 
may be an important finding. The computer, insensitive to the increase 

in disengaged behavior, is not flexible enough to employ a strategy 
designed to renew the student's interest in the activity. A teacher, 

therefore, should be aware of the implications of a slight increase in 
disengaged behavior and regard it as a warning that a total loss of 
engagement may be approaching if something is not done to rekindle the 
student's interest in the activity. 

Conclusions 

'i 

In conclusion, it may be argued that in our hypotheses we did not 
give enough importance to the engaging properties of the machine itself. 

If this is so, in our time-line conceptualization of the engagement process 
we should find that the properties of the machine that are important in 
providing initial engagement are also important in sustaining engagement, 
because they constantly recall attention to the stimulus source. It is 
important to keep in mind the complexity of the machine. The combina- 
tion of sound, motion, and fast pacing may encourage a high level of 
engagement at all times. 

We still predict a decrease in engaged (or an increase in disengaged) 
behavior as the programs become more and more difficult, eventually sur- 
passing the student's level of competence. We also predict that there will 
be a level of ease at which the student's engagement decreases and/or 
disengagement increases. In the computer classroom, we observed a 
qualitative difference in the disengagement resulting from a lesson that 
is too difficult and a lesson that is too easy. On lessons that were 
too difficult the students slumped down dejectedly with their heads on 
the teletypes. On lessons that were too easy, restless behavior pre- 
dominated. 
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One flaw in the study may have been the isolation of the subjects 
while they were working on the experimental lessons. There are frequent 
child-child and child-adult interactions in the computer classooom, such 
as (a) requests for help from a neighbor or the proctor, (b) social 
interchanges, or (c) venting of frustration on a neighbor, e.g. by kicking. 
This may be one reason for the consistently high engagement scores 
recorded during the experiment. Unquestionably, however, engagement is 
also high in the computer classroom. 

We may draw a few more tentative conclusions from the results of our 
study. Engagement and disengagement appear to be independent m.asures; 
disengagement proved to be the more sensitive measure in this study. 
Affective behavior seemed to have no relation to engagement and dis- 
engagement, but it may be a factor in classroom engagement. And, 
finally, engagement was high on easy lessons and did not begin to drop 
until the subjects missed more than 20 percent of the problems on 
each lesson. 
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